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ON THE EXPECTATIONS OF MEAN SQUARES BASED 
ON NONINDEPENDENT VARIATES IN FACTORIALS 



THIS DOCU^/EN^ ;^''^„pcetvED ^«ow^ 



John F. Draper 
CTB/McGraw-Hill 



A paper presented at: the 
American Educitfonal Research Association meetings in ChIca^',o 

Af)riJ ]97/( 



On the Expectations of Mean Squares Based 
on Noni.ndcpendont Variates in Factorials 

John F. Draper 

CTB /McGraw-Hill 

Several procedures have been published for aiding an investigator who 
wishes to perform an analysis of variance (iVNOVA) on data which can be 
assumed to have zero pairwise covariance (i.e., Cornfield & Tukey, 1956 and 
Millman & Glass, 1967). Others have published methods for modifying basic 
MOVA procedures so that they may be employed on data which have nonzero 
pair\>;ise covariance between data elements associated with the levels of one 
factor in a factorial experimental design (i.e., Greenhouse & Geisser, 1959) 
In all of these procedures it is essential to determine the expectation of 
each of the mean squares associated with sources of variation in the design 
both under null effect and non null effect conditions. Only if these expec- 
tations arc known can any appropriate variance ratio tests of effects due 
to a source of variation be determined. For in order to assume a central F 
distribution for a variance ratio test statistic whicli is testing a null 
effect, it is necessary to assume with assurance that the numerator and the 
denominator have identical expectations under conditions of null effects 
associated with the source of variation being tested, 

Af it will be shtn-.'n with an example later in tliis paper, there are case 
of factorial exper im^Mital designs in which a variance ratio test statistic 
has identical expectations for numerator and denominator under nuJl effect 
conditionj; wi u-n basrd on uncorruJatod variates but v/h.irh has unequal expect a 
tlons for nwucriiLor and dcnonilnaLor under null effect conditions v/hen based 
on varJates v.'hosc pai.rwisc covariances are not all identically zero, lliat 
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is, a test statistic which is unbiased when based on uncorrelated variates 
is biased when based on correlated variates. For this reason it is essen- 
tial, if an /VNOVA is being considered, to be able to deterraine the efUect of 
nonzero covariance among the variates on the expectation of mean squares in 
factorials . 

It is relatively well known that the n~dimcnsional array of variates for 
an n~way factorial experimental design may be mapped into a one-dimensional 
array or vector, (i.e., Y = _z, the n~diraensional array Y is mapped into the 
v^^ctor z) and that the AiN'OVA linear model may then be written in matrix form 
as 



where X is a "design matrix," is a vector of effect parameters, and e^ is a 
vector of error variates. 

If X iJ a full design matrix and is written in terms of zeros and 
unities, it will be of deficient column rank. Therefore, unique values of 
the elements of X eannot be determined. However, if appropriate contrasts 
are employed for the sources of variation, a reparameterized design matrix of 
full column rank, K, may be specified and X factored into the matrix product 



(1) 



2 = X Y + e 



KL, 



where 



(2) 



X = KL 



and where 



(3) 



L = (K'K) -^K'X . 
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Then a new linear model for the AI^OVA in terras of the coiiLrasts may be written 
as 

(4) = KL Y + e^ 

or, say 

(5) 2, = K 0^ + e 

where _0 = ljY and contains a new set of effect parameters v/hich are linear 
combinations of the parameters in Y* 

Tlie columns of K may be partitioneu into sets of columns such that 
each partition is associated with a source of variation in the ANO^^A and 
the sum of squares associated with that source of variation can be expressed 
as a quadratic form involving that partition of K and _z. For example 

(6) SS^ = (K'^^K^; -iK'^^z 



or sav 



(7) SS^ = 7/ z, 



v/hcrc F^^ stands in for the idcmpotent matrix of the form. 



The expccLaL ion of -ihe mean square associated with tlic A source of 
variation, /-.(MS^) , is equal to the expcctaLion of the svim of squares, L'(SS^) 
divided by thi'. appropriate, de^^rees of freedom, 
(B) K(MS^) = df-1 /^^(SS^) . 



Tne cxju'ctiaL ion for the sum of squares c/jp he written la terms of tlie 
quadratic form as 
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(9) ff(SS ) -El I f z 7 , 

' i=l j=l ^ J 

or removing the constants from the expectation 

n n 



(10) ixss^) 



y I f.. ff(z.z.) . 
i=l j=l -J ' ^ J 



Since tlie covuriance of z. and z. • a... is expressed by the identities 

1 3 ^3 



(11) a..=£'(z.2.) - E(z.) E(z.) 



and 



(12) E(z.z.) = ff(z.) ff(z.) + 0. 

13 1 J 



The ^^xpression for the expected sum of squares may be rewritten as 

(13) ff(SS^) = f I f (L'(z.) £'(r ) + a ) 

i=l i=l J J 



or as 

(14) ff(SS ) = ? 'J f i'Cz ) 2?(z ) + ? I f.. a 

i=l j=l i=l j=l -J ^ 



If the covariances are all zero then the 

n n n 

(15) s(ss,) = y f.. £'(z.) £'(z.) + y f.. 0.. 



Howcvcnr, if the covariancct; arc not till zero, Lho term 

(n-1) n 
(16) 2 ): I f a 

i=l j = i+l ^-^ 
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must ba added to the expression which obtains under the usual assumptions of 
zero covariance. 

What has been saora up to this point is that the determination of the 
expectation of mean squares based on variates with nonzero covariance can be 
accomplished by oniploying the same methods which apply when zero covariance 
is assumed, if a term ir -/olving the sum of products of the off diagonal 
eleiDcnts of an idcmpotent matrix of the form and the off diagonal elements 
of the covariance matrix is added. 

Hie development of this paper so far could be useful but is somewhat 
computationally inconvenient. Fortunately, further development results in 
considerable simplification, both computationally and conceptually. 

It will be shown that an idempotent matrix of the form, for a given 
source of variation, is a patterned matrix and has elements which come from 
a small set of easily determined constants, and that because of the way in 
which it is patterned the ranges of sujnmation under v;hich the elements do 
not change are also easily determined, for most designs. Note, that for 
ranges of the double sunmiation, 

n n 



(17) I I 




v;ithin v.'h Lch f,. does not change in Vt':luc, the f., constant may be removed 
from t'no sunimatioa as, 

(IB) f n • 

J ^ J J 

If is allov;t^d to represent the expectation of the mean square, /'/(MS^) 
given no nonztM'o ct>var i.aneu aiiiuiig tlici variates on whicfi it is based, and 
there ar(! k ranj'.es of douhJ.C! r^unmuilion v/i.th.in wh U:h f., do(^s nol ch«'in^',e, 
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then an expression for the expected mean square, i'CMS^), given some non^.ero 
covariance among the variates can be written as, 

k 

p=l ^ * P 1 J ^ 

range 
p of i 
and j 

where n is the number of elements in the p^^ ranee of sumraatiou, f is the 
P ^ * p 

th 1 V V 

constant value of f.. in the p range, and ^ I I o , , ±s the average 
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th r • 1 . 

covarj.ance in the p range of i and j . 



Or if the product n^ f^ divided by the degrees of freedom is considered 
as a coefficient for the p^^^ average covariance term, 

(20) . c^^ - df-^ n^ f^ , 



P P 



and the p^^^ average covariance is indicated by o^, then the general equation 
for the expected mean square can be expressed as 

(21) £'(>V = S % • 

p-1 



Tlie above expression is the basic formula toward which the preceeding 
develo[)jiit'n t has been directed. With it and a knov;led';e of how F is patterned, 
a concise expression for the expectation of a mean square can he determined 
in terms of a fann'.llar expression C^, plus a weij^htcd sum of certain avera^»e 
CO variances . 

Jt v;lll be useful nov; to consider how to spec i.ty the unique elements of 
a patterned "K" ichMiiju) ten t matrix and v/here tliey arci in it. In order to do 
this, first consJcUM; the nntuin* of a p;irtlllon of a rep:ir/imeler 1 zed des.Ij',n 
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matrix associated with a givun source of variation since an "F'^ matrix is 
a function of such partitions 



For any completely crossed and balanced n~way factorial design, the ''K" 
partition associated with any source of variation can be expressed as the 
Kronecker or direct product of n full column rank matrices each respectively 
associated with a factor of the factorial. The n full column rank matrices 
arc ordered from left to ri^^ht in the product in order of increasing "rapid- 
ity of subscript change"^ for each factor in the n-dimensional '*Y" array 
elements from which the 7^ elements v;ere mapped. For example, if the source 
A was in a 3-way factorial and its "subscripts changed most slowly," mif;ht 
be equal to 



For any completely balanced design v;ith nested factors a combination of 
two or more partitions obtained from a full rank desi^^n matrix, created as 
thou|;h tlie factors; aiM'c crossed, can be used to struct: 11 re the <'V|>tpropr iate 
idempotcm matrix of tiie form for a nested source of variation. For example, 
say an lii/itrlx is desired for the source, A nested v/Ith B. The "F" matrices 



(22) 



(e.g 



• 1 



A A A A^ A ^ 




where 1 is a vector of q unities. 



J 

Se.e Appendix A for explanation. 
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for the A source of variation and the A by B interaction source of variation 
(assuming A and B were crossed) may be added to form the "F" matrix for the 
A within B source. 
That is 

'^A:B = ^A ^AB ' 

since 

or in terms of a sum of quadratic forms 

(26) SS^^^ = ^A^^^' ^'ab^ ' 
therefore 

(27) = (F^ + F^^) z , 
and 

(28) = z- F^ , z . 

Of course unbalanced desi^s require partitioning of sets of rows with- 
in the column partitions and are thus more complicated but tractable, if not 
simple . 

Now as an example, consider the "F" matrix for tlie three-way interaction 
A by B by C in a 3-way completely crossed factorial. 
<29) K^, = A 9 B 0 C 

£ind 

(30) F = ( A 0 B G C ) [( A G B G C ) " ( A G B G C ) ( A 0 B 0 C ) " 

(31) = ( A 0 0 C ) [A'A 0 B"B 0 CCJ-^^ ( A G B 0 C ) 

(32) = ( A 0 B 0 C ) [(A"A)"1 G (B''H)-1 0 (C"C)~*^] ( A G B O C ) 

(33) = [A (A"A)-^ A"] 0 [B (B^B)"^ B"] G [C (CC)"! C] . 

In a slmil.ir manner any id empo tent "F" matrix as.sociateci with a source of 
vai.'iaLlon in a baiaut:<.'cl ii-v/ay i'.'jetor ial desi;',n can be ctxp l ess cd as a ICroneckc: r 
product of n .'■.c]>arate id<Miipotent matrices. Nov/ tlu* contrarits of the levcij.s 
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of any factor may be spccifiGtl so that, without any loss of gencraliiiation 
with respect to an overall ANOVA test of affects, there are at most two 
unique elements in an idempotent matrix such as 

(34) A (A^A)-i A^ . 

Tills means that an n-way balanced factorial will have patterned "F" matrices 
with a number of unique elements less than or equal to 2^. 

Now 1 will present an er.ample from which I hope generalization will be 
apparent since I have not yet set up any "rules of thumb" for these proce- 
dures. Consider agviln a three-v;ay factorial. Let the factors be completely 
crossed, balanced (n = 1) and be labeled A (for subjects) with levels 
i = 1,2, s; B (for occasions of measurement) with levels j = 1,2, r; 

and C (for measures) with levels k = 1,2, t. Further let factors A and C 

provide random sources of variation and B a fixed source of variation. A 
linear model for the dependent variates in a three-dimensional array may be 
written as 

(33) Y.., = M + a. + + -fd^ + f., -fg., +h,.. -fe... 

X3k 1 J k ij ik ^jk ijk ijk 

where 3^ , the effect duo to the j^^^ occasion, is the only fixed effect. Or 

if a vector v. is formed of the variates y... such that the three dimensional 

ij k 

array y is r.::i])ped int^) the one dimensional array 21> where 7/ = 



(30) 



y.) -1 -1 » • • • >y,. A » 



Llie linear iiiodol may In- v/ritten la (.ena.s of a dc;.<;Lgn maLi i';-: X, a vc:f:(:or of 
jja r;,iint'tc' r '<_ aiul a wxXvix: (;f (.Trur:; n_. 
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That is 

(37) .z = + £ > 



where e_ has elements corresponding; to the ^Ith subscripts ordered as 

they were f or y , . , , 
ij !<- 



(38) 



I - [^ii', i\ c^ d^ r, ^^ h^, and 



(39) 



X 



-srt 



I i 
IQ101 llQIGl 'lOlGI 



•r ~t I -s 



I 



1 I 

I0IG1 llGlOI IlGIQI 

s r ~L I s ~r c I ~s r t 

I r 



t : 



I 0 I G I 
s r t 



where 1 is a vector of n unities, and I is an identity matrix of order n. 
-n n 

Now for a i^ioment examine in Table 1 the sources of variation, degrees 
of freedom, an.l the expected mean squares associated with each source of 
variation under an assumption of independent variates as elements of ^, that 
is, assume z_ N (lliO^^^-l) under conditions of all null effects. 

Then consider how the additions to the E{yiS)s required under assump- 
tiont^ of noD/'.ero co variance are determined. 

In matrix not.'j.tion the linear model for the repa rame t eri^ cd ANOVA is 



(AO) 



- K 'h e_, v/luire 



(^^1) 



K - 



-srt 



I 



I 



I 



U C:J 1 0 1 I 1 0 V (') 1 I J 0 .1 G W ! U 0 V 0 1 I 
~i; -t 1 ~H -t -s -r I -t I 



U (•) 1 0 \'J I 
-r I 



I 



I 

Q V 0 VI i U G V 0 X 
I 
I 



1 



Tabic 1. 



SOURCES, dec;ri:i.:s of frekdom, and exi^ected mean squares for thi-: anova 

UNDER AN ASSUi-U'TiON OF ZF.RO COVARJAxN'CE .UIONC THE DATA EEEMENTS. 



Sourc e 
A 



E(MS) 



0^ + ro? + rto^ 



r-l 



I. 

o?- + a- + sa2 + to^ H- st(r-l)-l J (p, 
e n g d i 



t-1 



+ ra| + sra^ 
e f c 



AB 



(s-l)(i-l) 



o2 + ar + ta2 



BC 



(r-DCu-l) 



+ a?- + sa^ 



ABC 



(s-l)(r-l)(t-l) 



a?- + al 
e h 
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with tlici ci^iitrast: niaLrices U, V, and W ciach of full coluimi rank. 

Again v;ithout loi;s of ganQraJ.liiat.ion with rospoct to an overall ANOVA test 

for effects let, 



(42) 
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]. . . . 


. 1 


(A3) 
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• • 




V= 






. . 
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• • 

0 ... 0 
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0 . 


. . 0 
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1 
1 
1 
1 



~r 



and 



1 
-1 

0 



1 
1 
-2 

0 



0 0 



1 
1 
1 
-3 
0 



1 
1 
1 
1 
1 



-t 



Now the dia^',oaal clcnioiiLs of U (U^U)""-^ U'' arc all identically S*" (S-1) and 
off dia^^fjnal eici.HMit.s are all equal to -S""^. Vor the matrix V(V"V)"^ V the 
dia^^oaal and off d i a.:.;t)rial clcMr.ents are rcr.pcctively r^-^Cr-l) and -r*"-^ and for 
V/(VJ"W)""^ the two elcjrricnts are t*"^(t-l) and -t*"^. Vfnere the unity vectors 
are iuvulvi^l tiu^. ru'iuilt obLai.ns that ai.l of the e].(;nierits of J (1. ^1 1 
are i tU-u t i v:\ 1 l.y n"" ^ . 
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For this 3-way dL:si[;n any *'F" matrix is a "supor-super matrix" and has 
at most 2-^ = 8 unique elomcMii.s and thus Lhcrc arc ci.;lit ranges of sunuiuifion 
within which "f.." docs not chanr.o, no malter with which source of variation 
the "F" matrix is associated. The above-men Lioned summation ranges are 
coherent ranges and are specified in Table 2 by super matrix notation in 
terms of ihe overall variance- covariance matrix which is itself a super-super 
matrix as sho\^m in Figure 1. 
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Table 3 contains the eight values of unique elements of the '*F" matrices 
for each of the seven sourc(2s of variation indicated in Table 1, and Fi^;ure 1 
shows a model for the variance-covar iance matrix i-' which assumes zero covari- 
ance between elements involving different subjects. Let the average covariances 
defined by ranges 2, 3, and 4 in Table 2 be indicated respectively as Op , a^, 
and Oi^ (oi, is an average variance and 05, oq, 07, and 03 have been assumed to 
be identically zero). The symbols 02, 03, and 01+ represent respectively the 
average covariance between measures witliin occasions, the average covariance 
between the same measure on different occasions, and the average covariance 
between different measures on different occasions. 

The values in Table 3 and expression (20) were used to create coeffici- 
ents for the average covariances and the coefficients are tabulated in Table 4. 

Now by employing coefficients from Table 4 and the general equation for 
an expected mean square which was previously developed 



(45) (e.g. L^OV = 4^ + I -a J 

m=2 



for each of the sources of variation, the expected m.ean squares in Table 5 
can be determined. 
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Tabic 5. 



SOURCES, DEGIIHES OF l-RMEDCM, AND KXPECTED MEAN SQUARI-S FOR THE ANOVA UNDER 
THE ASSUMPTION OF NON-ZEKO COVARIiVNCE /UIONG DATA ELEMI^NTS OBTAINED FROM 

THE S/\>fE SUBJECT. 

Source dl E(MS) 



s-1 0"^ + r 

e 



ol + rta^ + (t-l)a2 + (r-l)a3 + (r-l)(t-l) 
1 a 



en g d ^ti J 

03 + (t-l)(02 - a^) 
t-1 0^ + ro^ + sro^ - 02 + (r-l)(o3 - o^) 



.05 ' (s-l)(r-l) 0^ + o^^ + to^ - ^3 + (t-1) (02 - o^) 



BC (r-l)(t-l) ol + + so| - 03 + (-1)(02 - oi,) 



ABC (s-l)(r-l)(t-l) ~ ^3 *^ ("D (c^2 - 
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Examine Table 1 and Table 5 and consider the formation of a variance 
ratio test ntatlstic to be employed in testing; the effects due to occasions. 
Usinc either table it appears that a Quasi-F ratio (Satteri:h*waite , 19A1) can 
be formed as 



(46) 



The Quasi-F ratio will at least have equal expectations for numerator and 
denominator even if an exact distribution for the statistic may not be 
known at present. 

Now examine Table 1 and Table 5 and consider testing the subject by 
occasions interaction source of variation. \^iile Table 1 provides the 
straight for^v'ard ratio 



(47) MS^3 



as a candidate, an examination of Table 5 indicates the expectation of the 
numerator is 



(48) o^^- -I- Oj.^- + t o^^ - 03 + (t-1) - 



and the cxpcclaLion for the denominator is 



(49) 
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Thus the two expectations difler by 



(50) t 0^^ -f t {02 - Ol,) . 



Miich means that if source of variation being tested, is zero, the 

numerator would still have an expectation which exceeded the denominator 
by t (02 - Oi^) . 



Therefore the ratio 



(51) MS^ 



could be a biased test statistic unless 02 = which would cause the term 
(52) f: (02 - Oi,) 

to vanish, but that could only result if the average covariance between 
measures on the same occasion was equal to the average covariance between 
different measures on different occasions, v;hich is unlikely. 

Implications 

The obvious imj^lication of the developments in this paper is that an 
invcsti;j;ator who collects complicated repeated measures type data should be 
very careful when it conu.is time to decide V7hat type of analysis is to be 
emi)ioyed. 'J'here is a wJde ran^.e of techniques from v;)u.ch to choose, from 
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a simple graphical displciy to complicated multivariate numerical iteration 
approaches, some of which provide maximum likelihood values along with the 
descriptive reorganization the others provide. 

Univariate A^OVA procedures however do have considerable appeal. The 
numerical procedures involved in an ANOVA and its base of inference are 
widely understood. Almost anyone can do one, and a large number of inves- 
tigators will not badJy misinterpret one. Fortunately the bias terms such 
as 

(53) t (02 - o^) 

can be estimated with assurance if the number of data elements is large and 
thus the mean squares can be "corrected" so as to form a ratio like 

(54) MS^ , t (§2 " 

ABC 

which would be an unbiased test statistic. And although T can't give the 
exact distribution of the above ratio when o^^ = 0 at present, I can report 
that my simulations indicate that the Box "conservative test" for the ratio 
(Box, 1954) .-appears to be indeed conservative. 

This means that a conservative investigator could use the relatively 
simple univariate ANOVA techniques and the developments of this paper to 
test hypotheses and make inferences from comj^l lea ted repeated measures type 
data . 
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Appendix A 



Rapidity of Subscript Change 

If the elnmcnts of an array of data for an n-way factorial design are 
subscripted with n subscripts, one for each factor in the design, where each 
subscript indicates the level of the factor with which it is associated, a 
coherent mapping of the elements of the array with n subscripts into an array 
of elements with one subscript requires a rule for subscript correspondences. 
The usual rule can be shown with an example if a five-way factorial is con- 
sidered with five-dimensional data array elements y. . , which are to be 

1,3 ,k,m,n, 

mapped into a one-dimensional array with data elements z^. If lower bound 
values of subscripts i,j,k,xn, and n are uniformly unity and the upper bound 
values respectively are specified as a,b,c,d, and e, then the correspondences 
for a balanced design can be shown by the formula 

p = (i-1) bcde + (j-1) cde + (k-1) de + (m-1) e + n 
or by the example 

^^1,1,1,1,1 
^1,1,1,1>2 
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^1,1 
^1,1 



^1,1 



^1,1 



1,1 



1,1 



1,1, e 
1,2,1 
1,2,2 

1,2,0 

l,d,e 

e,d ,e 

c: , d , i}. 



^e+1 



'e+2 



'2. e 



^d.o 



"c. d« e 



'a. b« c« d*ii 
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As can be seen from tlie excimple, the rightmost subscript of y changes most 
rapidly as the subscr.ipt for z clianges, and the next rightmost subscript of 
y changes next most rapidly and so on. 
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DeLcriiilnation of F ElemcnLs 

The values in Table 3 are easily determined once the pair of unique 
elements for each idompotcnt matrix in the Kronecker product expression for 
an "F" matrix antilogous to expression (33) are known. 



(33) F^^ = [A(A" A)-^ A"] Q [B(B" B)-l B"] Q [C(C" C)-l C"] . 



Let the main diagonal elements of an idempotent rr.j.rix in (33) be subscripted 
with a unity and let the off diagonal elements be subscripted with a 2, then 
the unique elements of the ^*F" matrix in the eight ranges are determined as 
sho\m belov.'. 

Range F Element 

1 ^l^^l'^l 

2 \'\'^2 

3 ^I'^Z'^I 

4 ^1*^2*^2 

5 ^2*^1*^1 

7 S'b2'^l 

from v/hich gcaio ralization can be seen. 

Note also J.n Table 3 that v;hen the "F" elements for the main effects 

!iave been determ lued , th(^ numerators of tiie elements for interacti^.i "F" 

iaatric:e5j can be formed from the products of Llie numeratorj; of the ^'involved" 
main effect "K" elements. 



